REALNET. 123 A PATENT 
VIDEO COMPRESSION AND DECOMPRESSION SYSTEM WITH 
POSTFILTER TO FILTER CODING ARTIFACTS 



5 Background of the Invention 

Field of the Invention 

The invention pertains to the field of video compression and decompression 
systems. More particularly, the invention pertains to a system and method for reducing 
compression artifacts in a video compression and decompression system. 

10 

Description of the Related Art 

Systems for applications of video and visual communications transmit, process 
and store large quantities of video data. To create a video presentation, such as a video 
movie, a rendering video system displays the video data as a sequence of individual 

15 digital images, also referred to as "frames," thereby simulating movement. In order to 
achieve a video presentation with an acceptable video quality, or to enable transmission 
and storage at all, the video systems process and modify the video data prior to 
transmission or storage. For instance, the video data is compressed and encoded to 
reduce the bit rate and the required bandwidth for storage and transmission of the 

20 images. 

In a conventional video system a video encoder is used to compress and encode 
the video data and a video decoder is used to decompress and to decode the video data. 
The video encoder outputs video data that has a reduced bit rate and a reduced 
redundancy. That is, the technique of video compression removes spatial redimdancy 

25 within a video fi*ame or temporal redundancy between consecutive video frames. In 

accordance with known image compression standards, such as MPEG, MPEG-2 and 
JPEG, an image coding process typically includes performing a block based frequency 
transform, e.g., discrete cosine transform (DCT), on an image to be transmitted. The 
resulting DCT coefficients are quantized or mapped to different quantization steps to 

30 render an approximate representation thereof If the available transmission bandwidth is 
relatively small, with respect to the complexity of the image to be transmitted, the size 




of the quantization steps needs to be relatively large. In that case, the resulting coarse 
quantization of the DCT coefficients introduces coding artifacts into the transmitted 
image and severely degrades the visual quality of the decoded sequence that may be 
displayed. 

5 Examples of such artifacts include mosquito artifacts and blocking artifacts. 

Mosquito artifacts are defined as temporarily nonstationary impulses that appear around 
objects which are moving within a decompressed video sequence. The mosquito 
artifacts result fi-om the coarse quantization of a prediction error signal. The majority of 
the energy contained in the prediction error signal is the result of a motion estimator's 

10 inabihty to distinguish between differently moving objects within the video sequence. 

For example, in videoconferencing applications the subject is generally against a 
stationary background. Since the motion estimator tries to match blocks of pixels 
between temporarily adjacent fi'ames, the boundaries between moving objects and 
stationary background that fall within these blocks cannot be detected. This leads to a 

15 situation where either a part of the background is assumed to be moving, or a part of the 
moving object is assumed to be stationary. If these prediction errors are coarsely 
quantized, impulsive artifacts result that change over time and tend to swarm aroimd the 
moving object, similar to a mosquito. 

Blocking artifacts are defined as the introduction of artificial block boundaries 

20 into the decoded video sequence. These artifacts are due to the combination of 
quantization and dividing the prediction error signal into blocks. That is, since there 
exists an inverse relationship between spatial extent and fi"equency extent analogous to 
the inverse relationship that exists between time and fi-equency extent in Fourier 
analysis, the quantization errors that occur in the DCT domain are smeared across the 

25 corresponding spatial block. Furthermore, since each block is quantized separately, the 

errors are most visible at the block boimdaries. 

In order to reduce the effects of the coding artifacts, it is known to apply a 
postprocessing technique to the recovered image. Since the artifacts typically comprise 
high firequency components, decoders in systems that apply such postprocessing include 

30 a postprocessor having a low-pass filter to filter out those components in the recovered 



image. However, the quality of the postprocessed image is dependent upon the selected 
parameters and may drastically vary from one set of parameters to another. 

Other systems use postprocessing filters that are spatially adaptive. These 
spatially adapted filters rely on local signal estimates and local noise power estimates to 
alter their responses. However, such an estimation of the noise power based on the 
quantization step size is not a reliable indicator as to the spatial location of mosquito 
artifacts and blocking artifacts within the decompressed video. For example, 
oversmoothing or blurring of the decompressed video occurs due to inaccurate estimates 
of the compressed video's signal-to-noise ratio. 

Thus, there is a need for a video compression and decompressing system and a 
method which suppress mosquito and blocking artifacts to improve upon the video 
quality a viewer perceives. 

Summarv of the Invention 

An aspect of the present invention involves a decoder apparatus for a video 
compression and decompression system having an input to receive an encoded video 
sequence and an output for a decoded video sequence. A video decoder is coupled to 
the input and decodes the received encoded video sequence. A filter module is coupled 
to the video decoder and the output and filters a decoded video sequence from the video 
decoder. The filter module has a variable filter strength that is a fimction of detected 
motion activity within the video sequence. 

Another aspect of the invention involves a filter module for a video compression 
and decompression system. The filter module has an input to receive a decoded video 
sequence and an output for the decoded video sequence. An activity counter determines 
motion activity within the decoded video sequence. A threshold detector is coupled to 
the activity counter and adjusts a filter strength as a fimction of the determined motion 
activity within the decoded video sequence. The threshold detector selectively adjusts 
the filter strength to one of a predetermined number of levels. 

A fiuther aspect of the invention involves a video compression and 
decompression system having an input to receive an encoded video sequence and an 
output for a decoded video sequence. A video decoder is coupled to the input and 



decodes the received encoded video sequence. A filter module is coupled to the video 
decoder and the output and filters a decoded video sequence from the video decoder. 
The filter module has a variable filter strength that is a function of detected motion 
activity within the video sequence. 
5 Another aspect of the invention involves a method of filtering a decoded video 

sequence in a video compression and decompression system. The method receives a 
decoded video sequence and determines a motion activity of each frame of the decoded 
video sequence. The method categorizes each frame as a frame of high activity or as a 
frame of low activity and adjusts a filter strength of a filter to filter the decoded video 
10 sequence as a function of the motion activity. 

Brief Description of the Drawings 
These and other aspects, advantages, and novel features of the invention will 
become apparent upon reading the following detailed description and upon reference to 
1 5 the accompanying drawings. 

Figure 1 is a high-level block diagram of a video compression and 
decompression system having an encoder apparatus and a decoder apparatus that 
includes a postfilter module. 

Figure 2 shows an exemplary embodiment of the postfilter module. 
20 Figures 3A and 3B are diagrams illustrating motion activity within a video 

sequence as a function of time. 

Figure 3C is a flow diagram illustrating the filter strength of the postfilter 
module as a function of time. 

Figures 4 is a diagram illustrating a procedure for varying the filter strength 
25 illustrated in Figure 3C. 

Detailed Description of the Preferred Embodiment 
In the following description, reference is made to the accompanying drawings, 
which form a part hereof, and which show, by way of illustration, specific embodiments 
30 in which the invention may be practiced. It is to be understood that other embodiments 
may be utilized and structural changes may be made without departing from the scope 
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of the present invention. Where possible, the same reference numbers will be used 
throughout the drawings to refer to the same or like components. Nimierous specific 
details are set forth in order to provide a thorough understanding of the present 
invention. However, it will be obvious to one skilled in the art that the present 
5 invention may be practiced without the specific details or with certain alternative 
equivalent devices and methods to those described herein. In other instances, well- 
known methods, procedures, components, and devices have not been described in detail 
so as not to unnecessarily obscure aspects of the present invention. 

Figure 1 is a high-level block diagram of a video compression and 
10 decompression system 1 (hereinafter "video compression system 1") having an encoder 
apparatus 2 and a decoder apparatus 4 that is coupled to the encoder apparatus 2 through 
a medium 6. The encoder apparatus 3 includes a video encoder 12 and a buffer 14. The 
^ decoder apparatus 4 includes a buffer 16, a video decoder 8 and a postfilter module 20. 

^1 The postfilter module 20 provides for a variable filter strength in accordance with the 

^ 15 present invention. The filter strength varies as a function of a determined motion 

fj activity within a video sequence in order to remove coding artifacts (e.g., mosquito 

artifacts and blocking artifacts) as explained below in greater detail. 

The encoder apparatus 2 encodes a video input sequence 8 (VIDEO IN) to 

•i I 

'Q generate an encoded and thus compressed representation in one of a number of possible 

iL; 20 formats. The format may be in an interleaved format tailored for "live" streaming of the 
3 encoded representation. The format may also be in a single file format in which each of 

the encoded representations are stored in a contiguous block within one file. This 
format is tailored for the "static" case in which a file is created for subsequent streaming 
by a server. 

25 The video input sequence 8 to the encoder apparatus 2 may be either a live 

signal, e.g., provided by a video camera, or a prerecorded sequence in a number of 
possible formats. The video input sequence 8 includes frames of a digital video, an 
audio segment consisting of digital audio, combinations of video, graphics, text, and/or 
audio (multimedia applications), or analog forms of the aforementioned. If necessary, 

30 conversions can be applied to various types of input signals such as analog video, or 
previously compressed and encoded video to produce an appropriate input to the 
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encoder apparatus 2. In one embodiment, the encoder apparatus 2 may accept video in 
RGB or YUV formats. The encoder apparatus 2, however, may be adapted to accept 
any format of input as long as an appropriate conversion mechanism is suppUed. 
Conversion mechanisms for converting a signal in one format to a signal in another 
format are well known in the art. 

The medium 6 may be a storage device or a transmission medium. In one 
embodiment, the video compression system 1 may be implemented on a computer. The 
encoder apparatus 2 sends an encoded video stream (representation) to the medium 6 
that is implemented as a storage device. The storage device may be a video server, a 
hard disk drive, a CD rewriteable drive, a read/write DVD drive, or any other device 
capable of storing and allowing the retrieval of encoded video data. The storage device 
is connected to the decoder apparatus 4, which can selectively read from the storage 
device and decode the encoded video sequence. As the decoder apparatus 4 decodes a 
selected one of the encoded video sequence, it generates a reproduction of the video 
input sequence 8, for example, for display on a computer monitor or screen. 

In another embodiment, the medium 6 provides a connection to another 
computer, which may be a remote computer, that receives the encoded video sequence. 
The medixmi 6 may be a network connection such as a LAN, a WAN, the Intemet, or 
the hke. The decoder apparatus 4 within the remote computer decodes the encoded 
representations contained therein and may generate a reproduction of the video input 
sequence 8 on a screen or a monitor of the remote computer. 

Aspects of the video compression system 1 illustrated in Figure 1 and described 
above can be combined and supplemented to achieve other embodiments. Numerous 
other implementations are consistent with the scope of this invention. Such other 
implementations need not be restricted to video, but may include audio or other forms 
of media as well. 

The video encoder 12 performs, for example, a discrete cosine transform (DCT) 
to encode and compress the video sequence 8. Briefly, the video encoder 12 converts 
the video input sequence 8 from the time domain into the frequency domain. The 
output of the video encoder 12 is a set of signal amplitudes, for example, called "DCT 
coefficients" or transform coefficients. A quantizer receives the DCT coefficients and 



assigns each of a range (or step size) of DCT coefficient values a single value, such as a 
small integer, during encoding. Quantization allows data to be represented more 
compactly, but results in the loss of some data. Quantization on a finer scale results in a 
less compact representation (higher bit-rate), but also involves the loss of less data. 
Quantization on a more coarse scale results in a more compact representation (lower bit- 
rate), but also involves more loss of data. 

The pre-existing video encoding techniques typically break up a frame (picture) 
into smaller blocks of pixels called macroblocks. Each macroblock can consist of a 
matrix of pixels, typically a 16x16 matrix, defining the unit of information at which 
encoding is performed. The matrix of pixels is therefore referred to as a 16x16 
macroblock. These video encoding techniques usually break each 16x16 macroblock 
further up into smaller matrices of pixels. For example, into 8x8 matrices of pixels or 
4x4 matrices of pixels. Such matrices are hereinafter referred to as subblocks. In one 
embodiment of the present invention, a 16x16 macroblock is divided into 4x4 
subblocks. Those skilled in the art will appreciate that the present invention is equally 
applicable to systems that use 8x8 subblocks, 4x4 subblocks or only 16x16 marcoblocks 
without breaking it up into subblocks. 

Fiuther, the pre-existing encoding techniques provide for motion compensation 
and motion estimation using motion vectors. The motion vectors describe the direction, 
expressed through an x-component and a y-component, and the amount of motion of the 
16x16 macroblocks, or their respective subblocks, and are transmitted to the decoder as 
part of the bit stream. Motion vectors are used for bidirectionally encoded pictures (B- 
pictures) and predicted pictures (P pictures) as known in the art. 

The buffer 14 of the encoder apparatus 2 receives the encoded and compressed 
video sequence (hereinafter "encoded video sequence") from the video encoder 12 and 
adjusts the bit rate of the encoded video sequence before it is sent to the medium 6. 
Buffering may be required because individual video images may contain varying 
amounts of information, resulting in varying coding efficiencies from image to image. 
As the buffer 14 has a limited size, a feedback loop to the quantizer may be used to 
avoid overflow or underflow of the buffer 14. The bit-rate of the representation is the 
rate at which the representation data must be processed in order to present the 



representation in real time. A higher bit-rate representation of the video input sequence 
8 generally comprises more data than a lower bit-rate representation of the same 
sequence. The most pertinent application of the bit-rate measure is in determining the 
rate at which the data of the representation should be streamed in order to present the 
representation in real time. Real-time playback of a higher bit-rate representation of a 
video sequence requires that the data of the representation be streamed and/or decoded 
at a faster rate than that required for a lower bit-rate representation of the same 
sequence. Thus, when streaming for real-time playback, a higher bit-rate representation 
requires a higher bandwidth communication link than a lower bit-rate representation. 

The decoder apparatus 4 performs the inverse function of the encoder apparatus 
4. The buffer 16 serves also to adjust the bit rate of the incoming encoded video 
sequence. The video decoder 18 decodes and decompresses the incoming video 
sequence reconstructing the video sequence and outputs a decoded and decompressed 
video sequence 24 (hereinafter "decoded video sequence 24"). In addition, the decoder 
apparatus 4 includes the postfilter module 20 that removes coding artifacts such as 
mosquito artifacts and blocking artifacts. The causes for these coding artifacts are 
explained above. The postfilter module 20 in accordance with the present invention 
reduces these artifacts without distorting the pictures which are output as a video output 
sequence 10. The video movie is therefore visually more pleasing for the viewers' eyes. 

Figure 2 shows the video decoder 18 coupled to an exemplary embodiment of 
the postfilter module 20. The postfilter module 20 includes an activity counter 22, a 
threshold detector 26 and a filter 30. An output 19 of the video decoder 18 is connected 
to an input 21 of the activity counter 22 and an output 23 of the video decoder 18 is 
connected to an input 29 of the filter 30. Further, the activity counter 22 is connected to 
an input 25 of the threshold detector 26 which is also connected to an input 27 of the 
filter 30. As discussed below in more detail, the input 21 of the activity counter 22 
receives a sequence of motion vectors, the input 29 of the filter 30 receives a bitstream 
representing the decoded video sequence and the input 21 of the filter 30 receives a 
filter strength control signal. The filter 30 outputs the video output sequence 10. 

It is contemplated that the postfilter module 20 is divided into the activity 
counter 22, the threshold detector 26 and the filter 30 for illustrative purposes only. 



Those skilled in the art will appreciate that such a division may not be required to 
implement the functionality of the postfilter module 20 in accordance of the present 
invention. For instance, the functionality of the postfilter module 20 may be 
implemented using a software module or a microprocessor that incorporates the 
functionalities of the components (22, 26, 30) shown in Figxu'e 2. 

The video decoder 18 receives a bit stream representing the encoded video 
sequence from the buffer 16 (Figure 1). In one embodiment, the video decoder 18 is a 
conventional MPEG decoder that includes a decoder controller, a VLC decoder 
(Variable Length Coding, VLC) and a reconstruction module. The operation and 
function of these components are known to those skilled in the art. These components 
are therefore described only to the extent believed to be helpful for a complete 
understanding of the present invention. For a more extensive description of a MPEG 
decoder, reference is made to generally available MPEG documents and publications. 
For instance, Barry G. Haskell et at, "Digital Video: An Introduction to MPEG-2," 
Chapman & Hall, ISBN 0-412-0841 1-2, Chapter 8, pages 156-182. 

The decoder controller receives the bit stream and derives control signals for the 
reconstruction module and the VLC decoder from the bit stream. Further, the decoder 
controller separates the encoded video data from the bit stream and inputs the encoded 
data to the VLC decoder. The decoder controller outputs control signals and status 
signals that include among others "block position," "frame select," "intra select" and 
"current picture" (not shown). 

The VLC decoder obtains from the encoded video data the quantized DC 
coefficients. Further, the VLC decoder obtains the motion vectors of each picture and a 
code word indicating the mode used to encode the video sequence (e.g., bidirectionally 
coded pictures (B-pictures) and predicted pictures (P-pictures)). In accordance with the 
present invention, the motion vectors are available for the activity counter 22 at the 
output 19 of the video decoder 18. 

The reconstruction module includes a dequantizer unit and an IDCT unit for 
calculating the inverse DCT. Using the encoded video data and the control signals 
provided by the decoder controller, the reconstruction module rebuilds each picture and, 
thus, creates the decoded video sequence 24 that is input to the filter 30. 



The activity counter 22 receives the motion vectors from the video encoder 18 
on a frame-by- frame basis. Each macroblock of a frame (e.g., a 16x16 macroblock) is 
further divided into subblocks, for instance, into 4x4 subblocks, wherein each 4x4 
subblock includes a motion vector representing motion of the 4x4 subblock, if any. The 
activity counter 22 uses these motion vectors to determine whether the presently 
analyzed frame (picture) is a high activity frame or a low activity frame. 

In one embodiment, the activity counter 26 includes a processor that analyzes 
these motion vectors of the subblocks. If either the differences of the x-components or 
the difference of the y-components of a motion vector with respect to the present 
subblock and the neighboring subblocks are greater than a predetermined integer the 
present subblock is categorized as an active subblock. That is, the predetermined 
integer is selected as a parameter to determine whether there is motion in the present 
subblock. The processor determines that there is motion when at least one of the 
differences (x-components, y-components) is greater than the predetermined integer. 
For instance, the predetermined integer is selected to be in a range between "1" and "3." 
In a preferred embodiment, the predetermined integer is "1." This procedure is 
repeated, for example, sixteen times and each 4x4 subblock is either in the active 
category or in the inactive category. The processor determines the number of active 
subblocks for each macroblock. 

If the number of active subblocks is greater than a predetermined threshold 
number, the processor characterizes this macroblock as an active macroblock. The 
predetermined threshold number is selected to define when a macroblock is active. That 
is, when a sufficient number of subblocks is active, the macroblock is defined as active. 
For instance, if the macroblock is divided into 4x4 subblocks, the predetermined 
threshold number is four so that the macroblock is active when more than four 
subblocks (i.e., 25%) out of 16 subblocks are active. In other embodiments, the 
macroblock may be defined as active when more subblocks are active (e.g., within a 
range between 30% and 50%), or when less subblocks are active (e.g., within a range of 
10% and 25%). 

The processor counts the number of active macroblocks for the whole frame and 
compares the number of active macroblocks to a defined threshold value. If the number 
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of active macroblocks is higher than the defined threshold value, the frame is a high 
activity frame (hereinafter "H" fi'ame.) If the number of active macroblocks is equal to 
or lower than the defined threshold value, the processor characterizes the fi'ame as a low 
activity frame (hereinafter "L" frame.) 
5 The total number of macroblocks in a frame depends on the size of the video 

frames or the resolution of the video frames. For instance, if the resolution is 640x480, 
the number of macroblocks is (640/16)x(480/16) = 1200. The defined threshold value 
may then be selected to be 300. That is, the frame is an "H" frame if more than 25% of 
the macroblocks are active. It is contemplated that the defined threshold value may be 
10 selected so that a frame is active when between 10% and 40% of the macroblocks are 
active. 

For each "H" frame the processor increases a counter and for each "L" frame the 
processor decreases this counter. A first counter value, therefore, increases and 
decreases as a fiinction of the "H" frames and the "L" frames. In one embodiment, the 
•^^ 15 processor implements two coimters. The first coimter operates as described. The 
?1 second counter, however, decreases with each "H" frame and increases with each "L" 

frame (Figure 3A). A second counter value, therefore, decreases and increases as a 
fiinction of the "H" frames and the "L" frames. 

i U 

Q Those skilled m the art will appreciate that, in another embodiment, the activity 

20 counter 22 may be implemented using a processor and separate counters coupled to the 
==3 processor. In yet another embodiment, the activity counter 22 may be implemented as a 

software module or a combination of a software module and firmware. 

The threshold detector 26 is coupled to the activity counter 22 and includes in 
one embodiment a comparator unit (e.g., including operational amplifiers). . The 
25 comparator unit receives, for example, the two counter values from the activity counter 

22 and determines the difference between the two counter values. The comparator unit 
compares this difference to predetermined threshold values as described below with 
respect to Figures 3 A, 3C. The threshold values may be determined through resistor 
circuits coupled to the comparator unit. In another embodiment, the threshold values 
30 may be stored in a programmable memory from which the comparator unit may read the 
respective threshold value. 
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The filter 30 is a digital filter that filters high-fi-equency components, such as 
pulses, firom the decoded video sequence 24. In accordance with the present invention, 
the filter 30 has a variable filter strength that depends upon the motion activity within a 
picture. The filter strength can be adjusted by varying the filter coefficients of the filter 
30. In one embodiment, the filter 30 may be adjusted to have one of a number of 
predetermined levels representing different filter strengths. In one embodiment, the 
filter 30 has three levels, Strong ("S"), Medium ("M") or Weak ("W") as shown in 
Figure 3C. It is contemplated that more than three levels may be defined. 

In one embodiment, the filter 30 may include or may be associated with a 
memory that stores the respective filter coefficients for the levels Strong, Medium and 
Weak. Upon the threshold detector 26 determining that the filter strength must be 
changed, the generated control signal (e.g., a code word corresponding to one of the 
levels) addresses the memory and the filter coefficients for the selected filter strength 
are loaded to the filter 30. 

Figure 3A is a diagram illustrating the motion activity within the video sequence 
24 as determined by the activity counter 26. As the activity counter 26 determines the 
occurrence of "high" and "low" activities as discrete events, Figure 3A shows two 
graphs 30, 32 representing these events as a continuous fimction of time (t). The graph 
30 is shown as a solid line and the graph 32 is shown as a broken line, wherein the 
graph 32 is a mirror image of the graph 30 relative to a horizontal axis through an offset 
value O. In another illustration, the offset value O may be zero and the horizontal axis 
may be the X-axis. Thus, starting at the offset value O, the graph 30 increases with a 
predetermined rate and the graph 32 decreases with the same rate. The graphs 30, 32 
intersect, for example, at t = t3, while the graph 30 is decreasing and the graph 32 is 
increasing. Figure 3 A fiirther shows difference values Al, A2, A3, A4 as described 
below. 

Figure 3B is a diagram illustrating a sequence of frames having high and low 
activities as a fimction of time (t). A fi-ame with a high activity is represented as "H" 
and a frame with a low activity is represented as "L." Figure 3B is aligned with Figure 
3 A to illustrate the increasing and decreasing graphs 30, 32 as a function of the "H" and 
"L" frames. The illustrated sequence of frames (from left to right) has five "H" frames, 
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ten "L" frames, eight "H" frames and eight "L" frames. Corresponding to these 
activities, the graph 30 in Figure 3 A increases during a period between t = tO and t = t2, 
decreases during a period between t = t2 and t = t6, increases during a period between t 
= t6 and t = t8 and decreases during a period between t = t8 and t = t9. 
5 Figure 3C is a diagram illustrating the filter strength of the postfilter module 20 

as a fimction of time. In one embodiment, the filter strength may be set to have one of 
three levels Strong ("S"), Medium ("M") or Weak ("W"). The postfilter module 20 is 
configured so that, as a default setting, the filter strength is at the level "M." From this 
level "M" the filter strength may change to one of the levels "S" and "W." For 
10 example, at t = tl, the filter strength changes from the level "M" to the level "S" and at t 
= t4, the filter strength returns to the level "M." At t = t5, the filter strength changes 
from the level "M" to the level "W" and at t = t7, the filter strength retums to the level 
^-3 "M." At t = t9, the filter strength changes again from the level "M" to the level "W." 

Hp 

H Figure 4 is a flow diagram illustrating a procedure 40 for varying the filter 

iy 

15 strength illustrated in Figure 3C. In describing the procedure, reference is made to 

Figures 2, 3A and 3B. During initialization of the procedure, the activity counter is 
R reset to its offset value O and the filter strength is set to be at the level "M" which is in 

1=^ one embodiment a default level as indicated in a step 42. 

Proceeding to a step 44, the procedure determines the motion activity within the 
20 sequence of frames. That is, the procedure determines whether the presently analyzed 
O frame is a "H" frame or a "L" frame, for example, as illustrated in Figure 3B. For each 

subblock within a 16x16 macroblock, the procedure looks at a present subblock and its 
neighboring subblocks and uses the differences of the motion vectors (x, y components) 
with respect to the present motion vector and the neighboring subblock to determine 
25 whether the present subblock is an active subblock or an inactive subblock This 

procedure is repeated sixteen times so that each subblock is either in the active category 
or in the inactive category. 

The procedure determines the number of active subblocks for each macroblock. 
A comparison of the nimiber of active subblocks with the predetermined threshold value 
30 is used to determine whether the macroblock is an active or inactive macroblock. In 
one embodiment, the predetermined threshold value is four. That is, if more than 25% 
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of the 16 subb locks are in the active category the macroblock is an active macroblock. 
It is contemplated that the threshold value may be set at a higher number. 

Proceeding to a step 46, the procedure determines the number of macroblocks 
for the whole frame and compares the number of active macroblocks to a defined 
threshold value. If the number of active macroblocks is higher than the defined 
threshold value, the frame is a "H" frame. In one embodiment, the defined threshold 
value is 25%, i.e., if the number of active macroblocks in a frame is greater than 25% of 
the number of macroblocks in the whole frame, the frame is an "H" frame. If it is a "H" 
frame, the procedure proceeds along the YES branch to a step 50, and if it is not a "H" 
frame, the procedure proceeds along the NO branch to a step 48. 

In the step 50, for each "H" frame the procedure increases a first counter and 
decreases a second coimter. As shown in Figure 3A, the first counter increases during 
the periods between t = tO and t = t2, t = t6 and t = t8 as shovra through the increasing 
graph 30, and the second counter decreases during these periods as shovra through the 
decreasing graph 32. These periods include the "H" frames as shdvm in Figure 3B, 

In the step 48, for each "L" frame the procedure decreases the first counter and 
increases the second counter. As shown in Figure 3A, the first counter decreases during 
the periods between t = t2 and t = t6, t = t8 and t = t9 as shown through the decreasing 
graph 30, and the second counter increases during these periods as shovm through the 
increasing graph 32. These periods include the "L" frames as shown in Figure 3B. 

Proceeding to a step 52, the procedure determines a difference A between events 
counted by the first counter and events counted by the second counter. Figure 3 A 
shows exemplary differences: a difference Al is shovm at t = tl, a difference A2 is 
shown at t = t4, a difference A3 is shovra at t = t5, a difference A4 is shovra at t = t7 and 
the difference A3 is shown again at t = t9. 

Proceeding to a step 54, the procedure determines if the difference A is positive 
and if the difference A is greater than a threshold value T(MS). In one embodiment, the 
threshold value T(MS) is an integer value, for example, eight (e.g., Al > 8). If the 
condition is satisfied, the procedure proceeds along the YES branch to a step 56. If the 
condition is not satisfied, the procedure proceeds along the NO branch to a step 58, 
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In the step 56, with the condition being satisfied, the procedure changes the filter 
strength fi-om the level "M" to the level "S" at t = tl as shown in Figure 3C. Once the 
filter strength is set to the level "S" the filter strength does not change unless another 
condition is satisfied. In Figure 3C the filter strength remains at the level "S" between t 
= tl and t = t4. 

In the step 58, with the condition being not satisfied, the procedure maintains the 
level "M" which is the default level for the filter strength. 

Proceeding to a step 60, the procedure determines if the difference A is negative 
and if the absolute value of the difference A is greater than a threshold value T(SM). In 
one embodiment, the threshold value T(SM) is an integer value, for example, five (e.g., 
abs (A2) > 5). If the condition is satisfied, the procedure proceeds along the YES 
branch to a step 64. If the condition is not satisfied, the procedure proceeds along the 
NO branch to a step 62. As indicated in the step 62, the procedure maintains the filter 
strength at the level "S." 

In the step 64, with the condition of step 60 being satisfied, the procedure 
changes the filter strength fi-om the level "S" to the level "M" at t = t4 as shown in 
Figure 3C. Once the filter strength is set to the level "M" the filter strength does not 
change unless another condition is satisfied. In Figure 3C the filter strength remains at 
the level "M" between t = t4 and t = t5. 

Proceeding to a step 66, with the filter strength being at the level "M" the 
procedure determines if the difference A is negative and if the absolute value of the 
difference A is greater than a threshold value T(MW). In one embodiment, the 
threshold value T(MW) is an integer value, for example, ten (e.g., abs (A3) > 10). If the 
condition is satisfied, the procedure proceeds along the YES branch to a step 70. If the 
condition is not satisfied, the procedure proceeds along the NO branch to a step 68. As 
indicated in the step 68, the procedure maintains the filter strength at the level "M." 

In the step 70, with the condition of step 66 being satisfied, the procedure 
changes the filter strength from the level "M" to the level "W" at t = t5 as shown in 
Figure 3C. Once the filter strength is set to the level "W" the filter strength does not 
change unless another condition is satisfied. In Figure 3C the filter strength remains at 
the level "W" between t = t5 and t = t7. 
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Proceeding to a step 72, with the filter strength being at the level "W" the 
procedure determines if the difference A is positive and if the difference A is greater 
than a threshold value T(WM). In one embodiment, the threshold value T(WM) is an 
integer value, for example, three (e.g., A4 > 3). If the condition is satisfied, the 
procedure proceeds along the YES branch to a step 76. If the condition is not satisfied, 
the procedure proceeds along the NO branch to a step 74. As indicated in the step 74, 
the procedure maintains the filter strength at the level "W." 

In the step 76, v^ith the condition of the step 72 being satisfied, the procedure 
changes the filter strength fi"om the level "W" to the level "M" at t = t7 as shown in 
Figure 3C. Once the filter strength is set to the level "M" the filter strength does not 
change unless one of the conditions is satisfied. In Figure 3C the filter strength remains 
at the level "M" between t = t7 and t = t9. At t = t9, the procedure changes the filter 
strength to the level "W" as described with reference to the step 66. 

Proceeding to a step 78, the procedure determines whether the sequence of 
fi-ames has ended (end of sequence "EOS"). If the sequence has not yet ended, the 
procedure returns along the NO branch to the step 54. Otherwise, the procedure ends at 
a step 80. 

The video compression system 1 and the method of filtering in accordance with 
the present invention provide for a reduction of mosquito artifacts and blocking artifacts 
without distorting the pictures of the a video output sequence 10 so that the video movie 
as a whole is visually more pleasing for the viewers' eyes. 

Furthermore, the postfilter module 20 is configured to implement a hysteresis 
for the levels of the filter strength. That is, the filter strength changes firom the level 
"M" to the level "S" at t = tl when the differences Al is positive and greater than the 
threshold value T(MS), but returns to the level "M" when a different condition is 
satisfied, namely that the difference A2 is negative and the absolute value of A2 is 
greater than the threshold value T(SM). Such a hysteresis avoids that the filter strength 
changes if the motion activity changes only briefly. 

Those skilled in the art will appreciate that the threshold values and threshold 
numbers referred to above are of exemplary nature. Different threshold values and 
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threshold numbers may be used to, for example, modify the hysteresis of the postfilter 
module 20. 

While the above detailed description has shown, described and identified 
several novel features of the invention as applied to a preferred embodiment, it will be 
5 understood that various omissions, substitutions and changes in the form and details of 
the described embodiments may be made by those skilled in the art without departing 
from the spirit of the invention. Accordingly, the scope of the invention should not be 
limited to the foregoing discussion, but should be defined by the appended claims. 

10 



□ 



-17- 



